Robotic drumming: synchronization in social tasks

Development of a rhythmic sense comes early in childhood development, and appears crucial for learning speech as well as music. Synchronized, rhythmic movements governed by oscillators play significant roles in industrial robots, but only in tightly controlled, precisely defined environments. Prior research focuses on the synchronization of the robot’s control system to either proprioceptive signals or external physical oscillators. Social tasks, however, require participants to detect, interpret and attune to the activity of humans, and by extension, to their intentions and perceptions. The synthesis of multiple sensory perceptions may be a fruitful approach to this problem. In order to evaluate this approach, we programmed a humanoid robot, Nico, to play a drum in concert with human drummers and at the direction of a human conductor. Our results show that sensory integration can enable precise synchronization in social tasks even when perceptual data is imperfect, misleading and subject to extensive processing delay. Through sensory integration, Nico can attune to a tempo that is set by a human conductor, in concert with human performers.